Section: Scientific Foundations

Statistical learning

Before detailing some issues of statistical learning, let us remind the definition of a few terms.

The fundamental tools used in SequeL come from the field of statistical learning [73] . We briefly present the most important for us to date, namely, kernel-based non parametric function approximation, and non parametric Bayesian models.

Kernel methods for non parametric function approximation

In statistics in general, and applied mathematics, the approximation of a multi-dimensional real function given some samples is a well-known problem (known as either regression, or interpolation, or function approximation, ...). Regressing a function from data is a key ingredient of our research, or to the least, a basic component of most of our algorithms. In the context of sequential learning, we have to regress a function while data samples are being obtained one at a time, while keeping the constraint to be able to predict points at any step along the acquisition process. In sequential decision problems, we typically have to learn a value function, or a policy.

Many methods have been proposed for this purpose. We are looking for suitable ones to cope with the problems we wish to solve. In reinforcement learning, the value function may have areas where the gradient is large; these are areas where the approximation is difficult, while these are also the areas where the accuracy of the approximation should be maximal to obtain a good policy (and where, otherwise, a bad choice of action may imply catastrophic consequences).

We particularly favor non parametric methods since they make quite a few assumptions about the function to learn. In particular, we have strong interests in $l_1$-regularization, and the (kernelized-)LARS algorithm. $l_1$-regularization yields sparse solutions, and the LARS approach produces the whole regularization path very efficiently, which helps solving the regularization parameter tuning problem.

Non–parametric Bayesian models

Numerous problems in signal processing may be solved efficiently by way of a Bayesian approach. The use of Monte-Carlo methods allows us to handle non–linear, as well as non–Gaussian, problems. In their standard form, they require the formulation of probability densities in a parametric form. For instance, it is a common usage to use Gaussian likelihood, because it is handy. However, in some applications such as Bayesian filtering, or blind deconvolution, the choice of a parametric form of the density of the noise is often arbitrary. If this choice is wrong, it may also have dramatic consequences on the estimation quality. To overcome this shortcoming, one possible approach is to consider that this density must also be estimated from data. A general Bayesian approach then consists in defining a probabilistic space associated with the possible outcomes of the object to be estimated. Applied to density estimation, it means that we need to define a probability measure on the probability density of the noise : such a measure is called a random measure. The classical Bayesian inference procedures can then been used. This approach being by nature non parametric, the associated frame is called Non Parametric Bayesian.

In particular, mixtures of Dirichlet processes [72] provide a very powerful formalism. Dirichlet Processes are a possible random measure and Mixtures of Dirichlet Processes are an extension of well-known finite mixture models. Given a mixture density $f\left(x\right|\theta )$, and $G\left(d\theta \right)={\sum }_{k=1}^{\infty }{\omega }_k{\delta }_{U_k}\left(d\theta \right)$, a Dirichlet process, we define a mixture of Dirichlet processes as:

where $F\left(x\right)$ is the density to be estimated. The class of densities that may be written as a mixture of Dirichlet processes is very wide, so that they really fit a very large number of applications.

Given a set of observations, the estimation of the parameters of a mixture of Dirichlet processes is performed by way of a Monte Carlo Markov Chain (MCMC) algorithm. Dirichlet Process Mixture are also widely used in clustering problems. Once the parameters of a mixture are estimated, they can be interpreted as the parameters of a specific cluster defining a class as well. Dirichlet processes are well known within the machine learning community and its potential in statistical signal processing still need to be developped.

Random Finite Sets for multisensor multitarget tracking

In the general multi-sensor multi-target Bayesian framework, an unknown (and possibly varying) number of targets whose states $x_1,...x_n$ are observed by several sensors which produce a collection of measurements $z_1,...,z_m$ at every time step $k$. Well-known models to this problem are track-based models, such as the joint probability data association (JPDA), or joint multi-target probabilities, such as the joint multi-target probability density. Common difficulties in multi-target tracking arise from the fact that the system state and the collection of measures from sensors are unordered and their size evolve randomly through time. Vector-based algorithms must therefore account for state coordinates exchanges and missing data within an unknown time interval. Although this approach is very popular and has resulted in many algorithms in the past, it may not the optimal way to tackle the problem, since the sate and the data are in fact sets and not vectors.

The random finite set theory provides a powerful framework to deal with these issues. Mahler's work on finite sets statistics (FISST) provides a mathematical framework to build multi-object densities and derive the Bayesian rules for state prediction and state estimation. Randomness on object number and their states are encapsulated into random finite sets (RFS), namely multi-target(state) sets $X=\{x_1,...,x_n\}$ and multi-sensor (measurement) set $Zk=\{z_1,...,z_m\}$. The objective is then to propagate the multitarget probability density $f_{k|k}\left(X\right|Z\left(k\right))$ by using the Bayesian set equations at every time step $k$:

where:

• $X=\{x_1,...,x_n\}$ is a multi-target state, i.e. a finite set of elements $x_i$ defined on the single-target space $𝒳$; (The state $x_i$ of a target is usually composed of its position, its velocity, etc.)

• $Z_{k+1}=\{z_1,...,z_m\}$ is the current multi-sensor observation, i.e. a collection of measures $z_i$ produced at time $k+1$ by all the sensors;

• $Z^{\left(k\right)}={\bigcup }_{t⩽k}{Z}_t$ is the collection of observations up to time $k$;

• $f_{k|k}\left(W\right|Z^{\left(k\right)})$ is the current multi-target posterior density in state $W$;

• $f_{k+1|k}\left(X\right|W)$ is the current multi-target Markov transition density, from state $W$ to state $X$;

• $f_{k+1}\left(Z\right|X)$ is the current multi-sensor/multi-target likelihood function.

Although equations (5 ) may seem similar to the classical single-sensor/single-target Bayesian equations, they are generally intractable because of the presence of the set integrals. For, a RFS $\Xi$ is characterized by the family of its Janossy densities $j_{\Xi ,1}\left(x_1\right)$, $j_{\Xi ,2}(x_1,x_2)...$ and not just by one density as it is the case with vectors. Mahler then introduced the PHD, defined on single-target state space. The PHD is the quantity whose integral on any region $S$ is the expected number of targets inside $S$. Mahler proved that the PHD is the first-moment density of the multi-target probability density. Although defined on single-state space X, the PHD encapsulates information on both target number and states. The Probability Hypothesis Density is a well-known method for single-sensor multi-target tracking problems in a Bayesian framework, but the extension to the multi-sensor case seems to remain a challenge.